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Nonnegative matrix factorization (NMF) has become a prominent technique for 
the analysis of image databases, text databases and other information retrieval and 
clustering applications. In this report, we define an exact version of NMF. Then we 
establish several results about exact NMF: (1) that it is equivalent to a problem in 
polyhedral combinatorics; (2) that it is NP-hard; and (3) that a polynomial-time 



< 

local search heuristic exists. 



> ■ 1 Nonnegative matrix factorization 

on ; 

Nonnegative matrix factorization (NMF) has emerged in the past decade as a powerful 
tool for clustering data and finding features in datasets. Lee and Seung [12j showed that 
NMF can find features in image databases, and Hofmann [10J showed that probabilistic 



latent semantic analysis, a variant of NMF, can effectively cluster documents according 
to their topics. Cohen and Rothblum [5] describe applications for NMF in probability, 
quantum mechanics and other fields. 

Nonnegative matrix factorization is denned as the following problem. The input is 
(A,k), where A is an m x n matrix with nonnegative entries, and k is an integer such 
that 1 < k < min(m, n). The output is a pair of matrices (W, H) with W G R mxfc and 
H G R fcxn such that W and H both have nonnegative entries and such that A « WH. 
The precise sense in which WH approximates A may vary from one author to the next. 
Furthermore, some authors seek sparsity in either W or H or both. Sparsity may be 
imposed as a term in the objective function 

The algorithms proposed by [TQl HH EG] and others for NMF have generally been based 
on local improvement heuristics. Another class of heuristics is based on greedy rank-one 
downdating [H [2], [31 [8] . No algorithm proposed in the literature comes with a guarantee 
of optimality. This suggests that solving NMF to optimality may be a difficult problem, 
although to the best of our knowledge this has never been established formally. 
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Waterloo, Ontario, N2L 3G1, Canada, vavasis@math.uwaterloo.ca. Supported in part by an NSERC 
award. 
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The main purpose of this paper is to provide the proof that NMF is NP-hard. This 
paper considers a particular version of NMF that we call exact NMF, which is defined as 
follows. 

EXACT NMF: The input is a matrix A G R mxn with nonnegative entries whose 
rank is exactly k, k > 1. The output is a pair of matrices (W, H), where W G R mxfc and 
H G R fcxn , W and H both have nonnegative entries, and A = WH. If no such (W, H) 
exist, then the output is a statement of nonexistence of a solution. The decision version 
of EXACT NMF takes the same input and gives as output yes if such a W and H exists 
else it outputs no. 

Implicit in the statement of exact NMF is an assumption that the rank of A is known. 
If A is specified as rational data, then its rank may determined in polynomial time via 
reduction to row-echelon form [6]. In practice, one would usually prefer singular value 
decomposition to determine rank(A) [9]. 

Observe that for any reasonable definition of the approximation version of NMF (that 
is, the version described earlier in which rank(A) is not constrained and in which one re- 
quires A pa WH instead of exact equality), an optimal algorithm when presented with an 
A whose rank is exactly k ought to solve the exact NMF problem. Thus, the "standard" 
NMF problem using any norm is a generalization of EXACT NMF. Therefore, any hard- 
ness result that applies to exact NMF (such as our hardness result) would presumably 
apply to most approximation versions as well. 

A different generalization of EXACT NMF is the problem of nonnegative rank deter- 
mination due to Cohen and Rothblum, which asks, given A £ R mxn with nonnegative 
entries, find the minimum value of k such that A = WH, W G R mxfe ) H G H kxn , and 
W, H have nonnegative entries. Cohen and Rothblum give a super-exponential time al- 
gorithm for this problem. Since nonnegative rank determination is a generalization of 
EXACT NMF, our result shows that it is also NP-hard. 

The proof of NP-hardness of EXACT NMF has two parts: In Section [2] we show 
equivalence between EXACT NMF and a problem in polyhedral combinatorics that we 
call INTERMEDIATE SIMPLEX, and in Section 1 we show the NP-hardness of this 
problem. A side result emerging from the proof of equivalence of EXACT NMF to IN- 
TERMEDIATE SIMPLEX is that certain local-search heuristic for EXACT NMF can be 
solved with linear programming (Section Hj). 

2 Equivalence to Intermediate Simplex 

In this section, we show an equivalence between the EXACT NMF and a problem in 
polyhedral combinatorics that we call INTERMEDIATE SIMPLEX. Although the focus 
in this section is on the decision version of these problems, it is apparent from the proofs 
that the search- versions could also be reduced to each other. The reductions use a number 
of arithmetic operations polynomial in m and n and are therefore polynomial-time for both 
the usual Turing machine model and the real-number model of Blum et al. [I]. 

A problem related to INTERMEDIATE SIMPLEX was proposed by Cohen and Roth- 
blum [5] and show to be equivalent to nonnegative rank determination. Therefore, their 
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results to some extent imply the results of this section. Nonetheless, we present the equiv- 
alence here in order to provide detail for our claim that all reductions are polynomial time. 

The equivalence is shown in three steps by first showing an equivalence to a problem 
denoted PI. 

PI : Given matrices Wo G R mxfc an d Ho G R fexn such that each has rank k and such 
that all entries of W H are nonnegative, does there exist a nonsingular matrix Q G H kxk 
such that WoQ' 1 and QH both have all nonnegative entries? 

Theorem 1. There is a polynomial-time reduction from EXACT NMF to PI and vice 
versa. 

Proof. First we demonstrate the reduction of EXACT NMF to PI. Suppose that we 
have an NMF instance, that is, a nonnegative matrix A of rank exactly k. In polynomial 
time (using, e.g., reduction to row-echelon form) one can factor A = W H such that 
W G R mxfe and H G R fexra . (This factorization does not solve exact NMF since the 
signs of the entries of Wq and Ho are unknown.) We claim that the original instance of 
EXACT NMF is a yes-instance iff the instance of PI is a yes-instance. For one direction, 
suppose the instance of EXACT NMF is a yes-instance, and suppose W, H are solutions to 
exact NMF. Then clearly Range(A) = Range(W^) = Range(W / ), which is a dimension-A; 
subspace of R n , and similarly Range(A T ) = Range(i? T ) = Range(iJ(f). This means that 
there exist two nonsingular kxk nonsingular matrices, say P, Q, such that W = WqP and 
H = QH . Thus, the equation WH = WoH may be rewritten as WqPQHq = WoH . 
Notice that W has a left inverse and H has a right-inverse since W has full column 
rank and H has full row rank. Thus, the previous equation simplifies to PQ = I (where 
/ denotes the kxk identity matrix), i.e., P = Q~ l . Thus, WqQ' 1 and QH both 
have nonnegative entries, so the instance of PI is a yes-instance. Conversely, suppose 
the instance of PI is a yes- instance. Then there exists Q such that W = \VoQ~ 1 and 
H = QHq both have all nonnegative entries, and WH = WqHq = A, so the instance of 
exact NMF is a yes-instance. 

For the opposite reduction, suppose we start with an instance (W , H ) of PI. Let 
A = WqH ; then A is nonnegative and has rank k. We claim that the instance of A is a 
yes-instance if and only if the instance of PI is a yes- instance. The proof uses essentially 
the same arguments as in the previous paragraph. □ 

In order to simplify the main proof in this section, it is helpful to define a slightly 
restricted version of PI as follows by requiring the last column of Wo to be all l's: 

RESTRICTED PI : Given matrices W G R mxk and H G K kxn such that (1) each 
has rank k; (2) all entries of WoH are nonnegative; and (3) the last column of Wo is all 
l's, does there exists a nonsingular matrix Q G H kxk such that WoQ^ 1 and QH both 
have all nonnegative entries? 

Theorem 2. There is a polynomial-time reduction from PI to RESTRICTED PI and 
vice versa. 

Proof. Given an instance (Wo, H ) of PI, we can produce an instance of RESTRICTED 
PI as follows. First, delete all rows of W that are identically O's. This does not affect the 
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rank of Wq, nor does it affect whether the product WoH is nonnegative. Finally, if Q is 
a solution problem PI prior to deletion of identically zero rows, then it is still a solution 
afterwards and vice versa. 

For the next step, let Q be a k x k nonsingular matrix chosen such that QH^e = e&. 
Here, e G R n denotes the vector of all l's, and G R fc denotes the last column of the 
k x k identity matrix. Such a Q is guaranteed to exist because H Q e cannot be zero: WoH e 
is the sum of columns of WqHq, which cannot be zero since the columns of WqHq are all 
nonnegative and WqHq is not identically zero by the assumption of rank at least 1. Then 
observe that (WqQ~ 1 , QH ) is a yes-instance of PI iff (Wq,H ) is a yes-instance. Such a 
Q may be found in polynomial time; for example, any k x k nonsingular matrix whose 
last column is HqG may be taken as and matrix inversion is polynomial-time in the 
Turing machine model [6]. 

Next, we observe that the last column of WqQ' 1 is W Q Q~ l ek = WqQ~ 1 QH e = 
W H e. We already argued above that this vector is nonzero, but now we will argue 
more strongly that every entry of WoH e is positive. First, note that WoH Q e is the 
sum of columns of the nonnegative matrix WqHq, and hence all its entries are at least 
nonnegative. Focus on entry i of WoH e; since it is a sum of nonnegative terms, then if 
it were zero then the entire ith row of WqHq would have to be zeros. This means that 
the ith row of Wq is orthogonal to every column of Hq. But since Hq has full rank, this 
is possible only if the ith. row of Wq is identically 0. However, this possibility is ruled out 
since we deleted identically zero rows of Wq. 

Thus, the last column of WqQ~ 1 contains all positive entries. Therefore, we can 
consider the instance of PI given by (DWqQ^ 1 , QH ) where D is an m x m positive definite 
diagonal matrix with diagonal entries chosen to make the last column of DWqQ~ x equal 
to 1. This instance of PI is a yes-instance only if the original instance was a yes-instance, 
because multiplying the first factor by a positive definite diagonal matrix does not affect 
the signs of WqHq nor of W Q~ l Q~ l . 

The opposite reduction, namely the one from from RESTRICTED PI to PI, is trivial 
since any instance of RESTRICTED PI is also an instance of PI. □ 

Now finally we get to the main new problem of this section. 

INTERMEDIATE SIMPLEX: We are given a polyhedron P = {x G R fc_1 : Ax > 
b} where A G R, nx ( fe - 1 ) and b G R n such that [A, b] has rank k. We are also given a 
set S C R fc_1 of m points that are all contained in P and that are not all contained in 
any hyperplane (i.e., they affinely span R fc_1 ). The question is whether there exists a 
(k — l)-simplex T such that S C T C P. 

Theorem 3. There is a polynomial-time reduction from RESTRICTED PI to INTER- 
MEDIATE SIMPLEX and vice versa. 

Proof. We will prove that both reductions exist at the same time by exhibiting a bijection 
between instances of RESTRICTED PI and instances of INTERMEDIATE SIMPLEX 
such that both directions of the bijection can be computed in polynomial time. 

Given an instance (Wq, H ) of RESTRICTED PI, we produce an instance of INTER- 
MEDIATE SIMPLEX as follows. The polytope P C R fe l is given by {x G R fe l : F (l : 
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k — 1, :) T x > —Ho(k, :) T }. (This constraint may be written more compactly as Hq[x.; 1] > 
0.) The set S of m points in P is given by S = {W (l, 1 : k — 1) T , . . . , W (m, 1 : k — 1) T }. 
The inverse mapping of this transformation starts with an instance of INTERMEDIATE 
SIMPLEX given by P = {x : Ax. > b}, A G R" 1 *^- 1 ) an d 5 = {xi, . . . , x m } and produces 
an instance of RESTRICTED PI given by 

w = ; ; 

V Xm 1 / 



and H = [A T ;-b T ]. 

We first show that all side-constraints present in the statement of RESTRICTED 
PI and INTERMEDIATE SIMPLEX are satisfied. The side-constraint that [A, b] has 
rank k is equivalent (under this bijection) to the side-constraint that Hq has rank k. 
The side-constraint that xi,...,x m affinely span R fc_1 is equivalent to requiring that 
[xx; 1], . . . , [x m ; 1] linearly span R fc , i.e., to the side-constraint that Wo has rank k. Finally, 
the side constraint that S C P means that Axj > b for % — 1, . . . , m, i.e., [A, — b] [sq; 1] > 
0, which is hence equivalent to the side-constraint that all entries of WqHq are nonnegative. 

We now show that the above bijection in both directions maps yes-instances to yes- 
instances. Let (S, P) be an instance of INTERMEDIATE SIMPLEX and (W , H ) the 
corresponding instance of RESTRICTED PI. Let T be a putative solution to the instance 
of INTERMEDIATE SIMPLEX. Let its vertices be g 1; . . . , g fc , which are vectors in R fc_1 . 
The condition that T C P is equivalent to requiring gi, . . . , G P, i.e., to H^^gi] 1] > 
for each i = 1, . . . , k. If we let 

G =( g i 1 ?)> w 

then we have shown that the condition T C P is equivalent to requiring HqG has all 
nonnegative entries. 

The condition that S C T means that for all i = 1, . . . , m, x$ G T. Recall that, by 
definition, a vector is inside a simplex if it is a convex combination of its vertices. Let 
be the putative vector of coefficients of the convex combination that expresses x$ in the 
hull of the vertices of T, for % = 1, . . . , m. In other words, 

[gi,...,gfc]qi = x<, (2) 

plus the requirements that the entries of q« are nonnegative and sum to 1. The latter 
constraint may be combined with (j2J) to write Gq.; = [x$; 1] where G is as in (JTJ, i.e., 
qi = Gr -1 W (i, :) T . The hypothesis that S C T is thus equivalent to the condition that 
each entry of G~ l W T for each i — 1, . . . , m is nonnegative, i.e., all entries of G~ l W Q T must 
be nonnegative. Hence, we have shown that T is a solution to the instance (S, P) if and 
only if G T is a solution to the instance (W , H ) of RESTRICTED PI. 

The argument is essentially the same in the other direction. Given an instance (Wo, Ho) 
of RESTRICTED PI, let (S, P) be the corresponding instance of INTERMEDIATE SIM- 
PLEX. Let Q be a putative solution to the RESTRICTED PI instance, and let gi, . . . , g^ 
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Figure 1: Illustration of Lemma [U The four large dots are the points in S; the thin solid 
line is the boundary of P, and the two triangles indicated with thick dashed lines are the 
two possible solutions T and T\. 

be the columns of Q T . Using the arguments in the previous paragraph shows that \VqQ~ 1 
and QHq have nonnegative entries iff S C T and T C P. □ 

An easy consequence of our transformation of EXACT NMF to INTERMEDIATE 
SIMPLEX is the observation that when rank(A) = 2, the NMF instance is always a 
yes-instance. The reason is that the resulting instance of INTERMEDIATE SIMPLEX is 
1-dimensional in which case P is an interval. However, if P is an interval then it is already 
a simplex, so one could take T = P to solve the instance. This observation yields a simple 
linear-time algorithm to find an exact nonnegative factorization of A case rank(A) = 2. 
case. This result was first established by Cohen and Rothblum [3], who also propose a 
simple linear-time algorithm. 

3 INTERMEDIATE SIMPLEX is NP-hard 

In this section, we will argue that the problem INTERMEDIATE SIMPLEX introduced 
in the previous section is NP-hard. 

Before delving into the statement of the main theorem and its proof, we first state the 
following simpler lemma and proof. This lemma describes the 'gadget' used in the main 
theorem below to encode a setting of a boolean variable. 

Lemma 1. Consider the following instance of INTERMEDIATE SIMPLEX: the poly- 
hedron P is given by P = {(x,y) G R 2 : < x, y < 1}, while the set S is given by 
{(0, 1/2), (1, 1/2), (1/2, 1/4), (1/2, 3/4)}. This instance has precisely two solutions T or 
T x defined by T = hull{(0, 0), (0, 1), (1, 1/2)} and T x = hull{(l, 0), (1, 1), (0, 1/2)}. 

A diagram of the lemma is given in Fig. [TJ It is easy to check that the side-constraints 
of INTERMEDIATE SIMPLEX (that S C P, that [A, b] has full column rank, that S 
affinely spans R 2 ) are satisfied by the above instance. 



Proof. The fact that S C Tj C P, i = 0, 1, is elementary to check. The fact that there 
are no other solutions is proved as follows. Suppose T is a solution. Let E and E\ be 
the two parallel edges of P given by Eq = {0} x [0, 1] and E\ = {1} x [0, 1]. Observe that 
the point (0, 1/2) 6 S lies on Eq, which means that the face of T containing (0, 1/2) must 
be either or 1-dimensional, and if it is 1-dimensional then it must be a subset of Eq. 
Similarly, (1, 1/2) must lie on a 0- or 1-dimensional boundary of T. It is not possible for 
both (0, 1/2) and (1, 1/2) to lie on 1-dimensional boundaries since a triangle cannot have 
two parallel edges. It is also not possible for both (0, 1/2) and (1, 1/2) to be 0-dimensional 
boundaries because in this case [0, 1] x {1/2} would be a bounding segment of T. Then 
all of T would have to be either above or below the segment, but then T would fail to 
cover either (1/2, 1/4) or (1/2,3/4), points in S. Thus, the only possibilities are (1) that 
(0, 1/2) is a vertex of T, and T has an edge that is a subset of E%, or (2) that (1, 1/2) is a 
vertex of T, and T has an edge that is a subset of Eq. But now one checks that in either 
case, in order to cover the two points (1/2,3/4) and (1/2, 1/4), the entire edge Eq or E\ 
must be taken as an edge of T. □ 

We now turn to the main result for this section, namely, the NP-hardness of INTER- 
MEDIATE SIMPLEX. In particular, we reduce 3-SAT [7J to this problem. Our reduction 
uses integers whose magnitude is polynomial in the instance of the 3-SAT instance, and 
hence our result is 'strong' NP-hardness. Recall that an instance of 3-SAT involves p 
boolean variables denoted xi, . . . , x p and q clauses denoted ci, . . . , c q . Each clause is a 
disjunction of three literals, where a literal is either a variable Xj or its complement Xj. 
An instance of 3-SAT is a yes-instance if and only if there exists a setting of the variables, 
that is, an assignment of a value of either or 1 to each variable, such that each clause 
is satisfied, i.e., at least one of its three literals is 1. It is assumed that the same variable 
does not occur twice (either in complemented or plain form) in any particular clause. 

Given such an instance of 3-SAT, we define the following instance of INTERMEDIATE 
SIMPLEX. It contains 3p + q variables (i.e., k — 1 = 3p + q) denoted Sj, Ui, i = 1, . . . , p, 
and Vj, j = 1, . . . ,q. These variables are written as (s, t, u, v) for short. The polyhedron 
P is defined by the following inequalities: 



P 



(s, t, u, v) : < s < u, 
< t < u, 
< u < e, 
v > 0, 

Si 2t^ Vj ■ • • • • * * ' / ^- ■ j 

2tj — 2s i — Ui < Vj whenever Xi G Cj 



whenever Xi ^ oj, 



> . 



(3) 



Here, e denotes the vector of all l's. In the above usage, e e R p , but we shall also use e 
to denote the vector of all l's in R 9 . 

Let ej denote the ith column of the identity matrix (either the p x p or q x q identity). 
The set of points S is defined as follows. Each of the points in the following equation is 
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also given a name for future reference. 




(e/(4p),e/(4p),e/(2p),2.5e/(8p)) 
(0, 0, 0, e,-) 
S={ (0,e,/4, ei /2,e) 




1. 
1. 
1. 

1. 



,Q 
,P 
,P 

,P; 
,P 



> . 



(4) 



(e i /2,e i /4,e i /2,e) 
(e i /4,e i /8,e i /2,e) 
^ 3^/8,^/2,6) 



Let us first confirm that the side-constraints of INTERMEDIATE SIMPLEX are satis- 
fied by this instance. Since G S, S affinely spans R 3p+9 iff it linearly spans H 3p+q . Points 
hj, j = 1, . . .,q, span the subspace defined by the last q coordinate entries. Fix some 
i G {1, . . . , p}. Subtract hi + . . . + h q from the three points r\, r?, r 3 . This yields three 
points whose nonzero entries are restricted to the (si,ti,Ui) positions; in these positions 
the three points have coordinate entries (0, 1/4, 1/2), (1/2, 1/4, 1/2) and (1/4, 1/8, 1/2), 
which are linearly independent. Thus, the subspace indexed by (si,ti,Ui) is spanned by 
S. This is true for all i, so therefore the points in S span all of H 3p+q . 

The next side-constraint is that the linear inequalities defining P are independent. 
One checks that the constraints s>0, t>0, u>0, v>0 imply that the constraint 
matrix contains a (3p + q) x (3p + q) identity matrix and hence has independent columns. 
We can also check that the right-hand side is independent of the columns of the matrix; if 
it were dependent, then there would be a point such that all the constraints are active at 
that point, which is obviously impossible (e.g., the constraints u > and u < e cannot 
be simultaneously active). The final side-constraint is that S C P, which is an elementary 
matter to check. 

The main theorem of this section is as follows. 

Theorem 4. The instance of 3-SAT is a yes-instance if and only if the above instance of 
INTERMEDIATE SIMPLEX is a yes-instance. In other words, the 3-SAT instance has 
a satisfying assignment if and only if there exists a simplex T such that S C T C P. 

Proof. First, let us choose some terminology for the coordinates of R 3p+9 . The individual 
coordinates may be denoted by s,, U, Ui or Vj for % = 1, . . . , p and j — 1, . . . , q. Collectively, 
the three coordinates (sj,tj,Mj) are called the "Xj coordinates" since they correspond to 
the ith boolean variable in the 3-SAT instance. 

Let T be a solution to the instance of INTERMEDIATE SIMPLEX. From T we will 
construct a satisfying assignment a for the 3-SAT instance. Clearly T has exactly 3p+q+l 
vertices. Observe first that the point is an extreme point of P and also lies in S, and 
therefore one vertex of T must be 0. 

Similarly, observe that each h^-, j — 1, . . . , q, lies on extreme edge of P, and therefore 
T must have q vertices of the form Xjhj, j = 1, . . . , q with each \ 3 , > 1. 

This accounts for all but 3p of the vertices of T. For an % G {1, . . . ,p}, let us say that 
a vector (s, t,u, v) G R 3p+IJ is Xi-supported if it is zero in all the Xj-coordinates for all 
j G {1, — {i}. More strongly, say that it is Xi-positive if it is a^-supported and is 

positive in at least one of the X{ coordinates. Fix a particular i G {1, . . . ,p} and consider 
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the four ^-points rj, . . . , rf which are all Xj-positive. Projected into the Xj coordinates, 
these points are (0, 1/4, 1/2), (1/2, 1/4, 1/2), (1/4, 1/8, 1/2) and (1/4, 3/8, 1/2). Since 
none of the T- vertices has negative entries, each of rj, . . . ,r| must lie in the hull only 
of T-vertices that are x^-supported such as 0, Aihi, . . . , X q h q . Furthermore, it must lie 
in the hull of at least one x^-positive vertex of T. In fact, there must be at least three 
such Xj-positive T-vertices since the four points, when projected into the Xj coordinates, 
are linearly independent. Thus, T must have at least three Xj-positive vertices for each 
% = 1, . . . ,p. Since there are only 3p vertices of T not yet enumerated, we conclude that 
T must have exactly three x^-positive vertices for each i, which we denote g^i, gj^, gi,3- 

Let gj,i, gi,2, &,3 £ R 3 denote the X; coordinates of gj,i, gi,2, g«,3- By the assumption 
that T covers the four points (0, 1/4, 1/2), (1/2, 1/4, 1/2), (1/4, 1/8, 1/2) and (1/4, 3/8, 1/2) 
in the projection into the Xj coordinates, we conclude that there must exist a 3 x 4 matrix 
B with nonnegative entries such that 

/ 1/2 1/4 1/4 
(&,i>gi,2,&,3)-fl= 1/4 1/4 1/8 3/8 
\ 1/2 1/2 1/2 1/2 

As mentioned above, all of g^i, gj i2 , gj,3 are nonzero. Because of the inequalities < s < u 
and < t < u that define P, it must be the case that the third entries of g^i, g ii2 , g ij3 are 
all positive and no smaller than the first and second entries. Therefore, define new vectors 
gi,i, gi,2, gi,3 that are all exactly 1/2 in the last coordinate and have other coordinates 
lying in [0, 1/2] obtained by rescaling each of gi,i, g«,2, gi,3 by twice its third coordinate. 
By rescaling B in a reciprocal manner, we find that there is a nonnegative matrix B such 
that 

/ 1/2 1/4 1/4 
(&,i>gi,2,&,3)- J B= 1/4 1/4 1/8 3/8 
\ 1/2 1/2 1/2 1/2 

By consider the third row of the above system of equations, we conclude that each column 
of B sums to exactly 1. Then dropping the third row on both sides yields the equation 

(g l , 1 (l:2),g,2(l:2),g l , 3 (l:2)).fi=( 1 ° 4 ^ , 

where the notation v(l : 2) denotes the first two entries of a vector. Now we observe that 
this is precisely a half-sized version of the instance of INTERMEDIATE SIMPLEX de- 
scribed in the preliminary lemma of this section, namely, find three points lying in [0, 1/2] 2 
whose convex hull covers the four points {(0, 1/4), (1/2, 1/4), (1/4, 1/8), (1/4,3/8)}. As 
established by the lemma, there are precisely two solutions to this system, which we 
will denote To/2 and Ti/2. Let Co be the set of i's such that the triangle defined by 
(gi,i(l : 2),gj )2 (l : 2),gi i3 (l : 2)) is To/2, while C\ is the set of i's such that this triangle 
is Ti/2. Thus we conclude that for i e C , 

(&,i> Si,2, &,3) = (^,i(0, 0, 1), /v(0, 1, 1), ^ ii3 (l, 1/2, 1)), (5) 

and for i G C±, 

(&,i> Si,2, gi, 3 ) = 0, 1), /ii, 2 (l, 1, 1), ^i, 3 (0, 1/2, 1)), (6) 
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where fi^k > for k = 1,2,3. This determines the Xi entries of gj^, k = 1,2,3, and 
the remaining Xj entries are zeros since gj^ is £j-positive. Therefore, it remains only 
to determine the Vj entries of g^*,, k = 1,2,3. There are several constraints on these 
entries as follows. First, we have the inequalities Vj > 0, and thus all those entries 
must be nonnegative. Next, we have the constraints Sj — 2U < Vj whenever and 
2ti — 2si — u.i < Vj whenever Xi G Cj. These inequalities are redundant whenever their 
left-hand side is nonpositive since we have already constrained Vj > 0. Thus, we need 
only consider the cases when the left-hand sides are positive. We see that the left-hand 
side of the first inequality Sj — 2U < Vj is positive only in the case of g^i only for i G Ci, 
and the left-hand side of the second inequality 2ti — 2si — u\ < Vj is positive only in the 
case of gi t 2 only for i G Co. Thus, for i G C±, for all j such that x« occurs as a literal in 
clause Cj, we must have 

gi.ik - A*»,i- ( 7 ) 
(Here, the notation g^iL denotes the Vj coordinate entry of gj,i.) Similarly, for i G Co, 
for all j such that xi occurs as a literal in clause Cj, we must have 



> 



Next, T must contain the point b from (jl]), so there must be coefficients a^, i = 
1, . . . ,p, k = 1, 2, 3 and 9j, j = 1, . . . , q adding up to at most 1 and all nonnegative such 
that 

P 3 q 

b = ^2 a hkSi,k + ^2 Oj^jhj. (9) 

i=l k=l 3=1 

Fix a particular %. The projection of b into Xi coordinates is b = (l/(4p), l/(4p), l/(2p)). 
Referring back to ([5]) and ([6]), one can see that regardless of whether % G Co or % G C\, b 
is expressed uniquely as b = (8p/J,i,x) + gi )2 /(8p/i i)2 ) + gi )3 /(4p^ ii3 ) . Therefore, 

a it i = l/^p/^i); a i>2 = l/(8p/^, 2 ); a i>3 = l/(4p/i i>3 ). (10) 

Suppose 2 G C . Then for each j such that Xj occurs as a literal in clause Cj, if we combine 
dU) and fllOl) . we obtain 

3 

^ «i,fcgi,fc 
fc=i 



> l/(8p). 



The identical inequality holds when i G Ci and Xj G cj. 

Now, sum the preceding inequality for i — 1, . . . ,p to obtain 



p 3 

i=l fc=l 



> ^/(Sp), (11) 



where mj is the number of literals X >'i Cj with i G Co plus the number of literals Xi G Cj 
with i G Ci. Let us now combine these inequalities: From (0J, b|^ = 2.5/(8p). From Qj, 

P 3 

b\ Vj > aj,kgi,k 

i=l k=l 
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since the last term of (Q is nonnegative. Finally, from (TTTT) . the above summation is at 
least rrij/ (8p). Thus, we conclude that rrij < 2.5. Since rrij is integral, this means rrij < 2. 
Let a be the setting of the x^s in the 3-SAT instance defined by taking Xi = 1 for i G C\ 
and Xi = for i G Co. Then if and i G Co, this literal is falsified in the clause. 

Similarly, if G Cj and z G Ci, then this literal is also falsified. In other words, rrij is the 
number of literals in clause Cj falsified by assignment a. We have just argued that rrij < 2 
for all j — 1, . . . , q. In other words, for each clause, there are most two literals falsified 
by assignment a. Therefore, a is a satisfying assignment for the 3-SAT instance. 

Summarizing, we have proved that if there is a simplex T solving the instance of 
INTERMEDIATE SIMPLEX, then there are exactly three vertices of T that are x r 
positive for each i = 1, . . . ,p; that, based on these vertices, i can be classified as either Co 
or C\\ and that the assignment a of the boolean variables in the original 3-SAT instance 
derived from C and C\ must be a satisfying assignment. 

Conversely, suppose the 3-SAT instance has a satisfying assignment. The vertices of 
T will be 0, Aih 1; . . . , \ q h q together with g,^, g i 2 , g«,3 for each i = 1, . . . ,p, defined as 
follows. Let Co index the variables set to by the satisfying assignment and C\ the 
variables set to 1. Define gi,i, gj,2, g«,3 as in ([5]) and according to C and C\. Take 
t^i,k — 5/8 for all (i, k). (Any other value slightly greater than 1/2 will work.) When 
i G Co and Xi IS cL literal in c,-, then take g^l^- = 5/8. When % G C\ and X% IS cL literal in 
Cj, then take gi,i|^ = 5/8. In all other cases, take gi t k\vj = 0. It is easy to see that all the 
inequalities defining P are satisfied by these choices. Furthermore, all the points in 5* are 
covered by convex combinations of the 3p + q + 1 points 0, A]hi, . . . , X q h q , g^, . . . , g p 3 , 
which are the vertices of T. 

For example, the point v\ = (0,e i /4 : ,e i /2,e) in the case that i G C is expressed as 
(2/5)gj i i + (2/5)gj.2 + h, where h is some linear combination of Aihj., . . . , X q h q chosen 
to make the Vj entries each equal to 1. (Note that the Vj entries of (2/5)g ij i + (2/5)gj >2 
before h is added will be either or 1/4). The total sum of the coefficients to express 
(0, ej/4, ej/2, e) is 2/5 + 2/5 + hi, where hi is the sum of the coefficients needed in the 
terms of h. Select Ai, . . . , X q to be large scalars so that we can be assured that 4/5+ /ii < 1. 
If this sum is less than 1, then we include a contribution of 0, another vertex of T, in the 
linear combination to make the sum of coefficients exactly 1. 

Similarly, as sketched out earlier, to obtain the point b = (e/ (4p), e/ (4p), e/ (2p), 2.5e/ (8p)) 
in the hull of the vertices of T, we use (Q with coefficients chosen according to (flQj) . This 
choice of a^j's yields coordinate entries equal to (l/(4p), 1/ (4p), 1/ (2p)) for each i and 
has entries less than or equal to 2/(8p) in each Vj coordinate entry. Then, as above, 
one can include additional terms involving and Aih 1; . . . , X q h q to complete the convex 
combination. One point to note is that the sum of the a^fc coefficients appearing in (Q, 
assuming ii^ = 5/8, is equal to 4/5, and hence does not exceed 1. Addition of the Oj co- 
efficients will make the total higher but still less than 1 provided Ai, . . . , X q are all chosen 
to be very large. □ 
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4 Local-search heuristics 



In this section we will prove a theorem about the INTERMEDIATE SIMPLEX problem 
that will suggest a class of local-search heuristics. The theorem is as follows. 

Theorem 5. Consider an instance of INTERMEDIATE SIMPLEX given by polytope 
P C R fc_1 with n facets and point set S C P with m vectors. Suppose there exists a 
solution T , and suppose that all vertices of T are given except for one. Then the set 
of feasible positions for the last vertex is defined by a system of linear equations and 
inequalities (mk equalities and n + mk inequalities). 

Proof. Let the vertices of T be denoted Vi,...,v fe , and suppose all are known except 
Vfc. Two sets of constraints must be satisfied, namely, those arising from the requirement 
S C T and those arising from T C P. Since the simplex T is assumed to be a solution, 
the given values of Vi, . . . , v^.i must all lie in P, and hence the constraint on to ensure 
that T C P is simply that v k G P. This clearly amounts to a set of n linear inequalities 
that must be satisfied by P. 

Next, consider the requirement S C T; choose a particular vector b e S. If b is in the 
hull of Vi, . . . , Vfc_! then b is in T no matter what choice is made for v^, so such a b does 
not impose any constraint on v^. Else suppose b is not in the hull of Vi, . . . , Vfc_i. Then 

the requirement on v k is that there exist Ai, . . . , A* such that AivH hAfcV fc = b, Aj > 0, 

% — 1, . . . , k, and Ai + • • • + X k = 1. This constraint is nonlinear because of the product of 
unknowns A^v^. However, we can rearrange it into a linear constraint by dividing through 
by Afc (which is nonzero by the hypothesis that b is not in the hull of v 1; . . . , v^-i) and 
defining new variables = Xi/Xk, i = 1, . . . ,k — 1, and a* = 1/Xk- Then the above 
constraints become ctiVi + • • • + ak-iVk-i + v fc = ®*h, ai > 0, i — 1 . . . , k — 1, a* > 0, 
a\ + • • • + ctfc-i + 1 = a*, which are all linear. There are k equality constraints and k 
inequality constraints in this system. A system of this kind is needed for each point in 
S. □ 

The preceding theorem suggests a local search heuristic for INTERMEDIATE SIM- 
PLEX. One can choose as an initial guess T a large simplex that contains all of S but 
perhaps is not contained in P. Then one adjusts the vertices of T one at a time, optimizing 
a criterion that minimizes departure of the vertex from feasibility. Because the feasible 
positions for the vertex under consideration form a polyhedron, several possible criteria 
such as 2-norm distance to feasibility would constitute convex programming problems. 
Thus, on each iteration of the local search algorithm, one could reposition a single vertex 
of T optimally until a solution is found. 
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